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(54) ROBOT AND INFORMATION PROCESSING SYSTEM 



(57) A robot provides a communicating means to 
move based on the transmission and reception of infor-- 
mation to/from the outside, thereby making it possible 
to realize a robot with high usability. In addition, it rec- 



ognizes the user's condition based on sensors provided 
around a user and outputs a response message accord- 
ing to the recognition. Thus, an information processing 
system which is very helpful can be realized. 
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Description 
Technical Field 

[0001] This invention is related to a robot and an in- 5 
formation processing system, and for example, is suit- 
ably applied to an entertainment robot treated as a pet. 

Background Art 

10 

[0002] Recently, information communication technol- 
ogy using information transmission lines, such as a tel- 
ephone line for public telephones, mobile telephones 
and the like, Internet, and communication satellite cir- 
cuits has been significantly improved in order to surely is 
transmit various kinds of information via images and 
sounds at real time: 

[0003] However, such the current information com- 
munication technology that can deal with only images 
and sounds has a problem that it lacks reality in the case 20 
where a user transmits his feelings to a communicating 
party, for example. 

[0004] Besides, in recent years, development and 
production of robots has been prosperous in addition to 
the data communication technology. For example, an 25 
amusement robot treated as a pet has been considered, 
which looks like a dog, a cat or the like and can auto- 
matically act according to the surrounding situations. 
[0005] Such the amusement robot, however, merely 
acts similarly to a dog or a cat, and can not act, for ex- 30 
ample, so as to be very helpful for a user, other than 
amusements. 

[0006] For example, if such a amusement robot has 
a communicating function for communicating with the 
outside through various kinds of transmission lines, 35 
such as a telephone line, Internet, and communication 
satellite circuits, a user can visually transmit his feelings 
to the amusement robot via movements using the com- 
municating function and can make the robot act so as 
to be very helpful for himself, so that it will be possible 
to improve the value of utilizing an amusement robot. 

Disclosure of Invention 

[0007] The present invention has been made in view 45 
of the aforementioned problems, and is intended to pro- 
vide a robot and an information processing system that 
are valuable in use. 

[0008] In order to solve the aforementioned problems, 
the present invention provides a robot comprising a so 
movement part for produce movements, an actuator for 
driving the movement part, an actuator control means 
for controlling the actuator, a communicating means for 
receiving signals used between the robot and another 
electronic device, and an information detecting means 55 
for detecting information from signals received by the 
communicating means. The actuator control means 
controls the actuator based on the inf ormation detected 



by the information detecting means, to possible to im- 
prove the value of the robot in use. 
[0009] Further, in the present invention, the robot 
comprises a memory means for memorizing stated in- 
formation, a memorized content changing means for 
changing the memorized content memorized in the 
memory means, a communicating means for receiving 
signals used between the robot and another electronic 
device, and a request detecting means for detecting an 
information updating request which indicates the update 
of stated information from the signals received by the 
communicating means. The memorized content chang- 
ing means updates stated information based on the in- 
formation updating request, which is detected by the re- 
quest detecting means, thereby it is possible to improve 
the value of the robot in use. 

[0010] Further, in the present invention, the robot has 
an output-of-sensor receiving means for receiving out- 
puts from one or more sensor(s) provided in a user him- 
self or near the user, a user's condition recognizing 
means for recognizing the user's condition based on the 
outputs of the sensor(s) received by the output-of-sen- 
sor receiving means, a movement part for producing 
stated movements, an actuator for driving the move- 
ment part, an actuator control means for controlling the 
actuator, and a telephone communicating means for re- 
ceiving signals used between the robot and another 
electronic device. When the telephone communicating 
means receives a communication signal from another 
electronic device, if the phone call is not answered within 
a fixed time, the actuator control means makes the 
movement part produce stated movements based on 
the user's condition recognized by the user's condition 
recognizing means, thereby it is possible to improve the 
value of the robot in use. 

[0011] Furthermore, in the present invention, an infor- 
mation processing system comprises one or more sen- 
sors) provided in the user himself or near the user, a 
user's condition recognizing means for recognizing us- 
er's conditions based on outputs of the sensor(s), a re- 
sponse message memory means for memorizing re- 
sponse messages according to the user's conditions, 
and a response message output means for outputting a 
response message. When a phone call is not answered 
within a fixed time, the response-message output 
means outputs a response message based on the us- 
er's conditions, which are recognized by the user's con- 
dition recognizing means, thereby it is possible to im- 
prove the value of the robot in use.. 
[0012] Furthermore, the present invention provides 
the robot comprising a movement part for producing 
movements, an actuator for driving the movement part, 
an actuator control means for controlling the actuator, a 
communicating means for transmitting signals used be- 
tween the robot and another electronic device, an infor- 
mation input means for inputting information from the 
outside, and an information output means for outputting 
information to the outside. The actuator control means 
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controls the actuator based on the information, which is 
input to the information input means, to information out- 
put means outputs the stated information from the com- 
municating means to another electronic device based 
on the information, which is input to the information input 
means, thereby it is possible to improve the value of the 
robot in use. 

[0013] Furthermore, the present invention provides 
the robot comprising a sensor means for inputting infor- 
mation from a user, a movement part for producing 
movements, an actuator for driving the movement part, 
an actuator control means for controlling the actuator, 
an information output means for outputting stated infor- 
mation, and a communicating means for transmitting 
signals used between the robot and another electronic 
device. When an input is not reached from a user within 
a fixed Time, the actuator control means makes the 
movement part produce stated movements. After pro- 
ducing the stated movements, when the input is not 
reached yet from the user within a fixed time, the com- 
municating means transmits stated information output 
from the information output means to another electronic 
device, thereby it is possible to improve the value of the 
robot in use. 

[0014] Furthermore, the present invention provides 
the robot comprising a movement part for producing 
movements, an actuator for driving the movement part, 
an actuator control means for controlling the actuator, a 
signal receiving means for receiving signals used be- 
tween the robot and another electronic device, an elec- 
tronic device control means for controlling another elec- 
tronic device, and an information detecting means for 
detecting stated information from the signals received 
by the signal receiving means. The actuator control 
means controls the actuator based on the stated infor- 
mation detected by the information detecting means to 
make the movement part produce stated movements, 
and the electronic device control means controls anoth- 
er electronic device based on the stated information de- 
tected by the information detecting means, thereby it is 
possible to improve the value of the robot in use. 

Brief Description of the Drawings 

[0015] Fig. 1 is a perspective view showing the struc- 
ture of a robot according to the present invention. 
[0016] Fig. 2 is a block diagram showing the circuit 
structure of the robot shown in Fig. 1 . 
[001 7] Fig. 3 is a perspective view showing the mech- 
anism of joints of the robot shown in Fig. t . 
[0018] Fig. 4 is a block diagram showing the circuit 
structure of the joints of the robot shown in Fig. 3. 
[001 9] Fig. 5 is a block diagram used for explaining a 
telephone function of the robot. 

[0020] Fig. 6 is a block diagram used for explaining 
the telephone function of the robot. 
[0021] Fig. 7 is a block diagram used for explaining 
the telephone function of the robot. 



[0022] Fig. 9 is a block diagram used for explaining 
the telephone function of the robot. 
[0023] Fig. 1 0 is a block diagram used for explaining 
the telephone function of the robot. 

5 [0024] Fig. 11 is a block diagram used for explaining 
the telephone function of the robot 
[0025] Fig. 12 is a schematic diagram used for ex- 
plaining an answering telephone function. 
[0026] Fig. 13 is a schematic diagram used for ex- 

10 plaining the answering telephone function. 

[0027] Fig. 14 is a schematic diagram used for ex- 
plaining a call connecting function. 
[0028] Fig. 15 is a schematic diagram used for ex- 
plaining a remote control function. 

15 [0029] Fig. 16 is a schematic diagram used for ex- 
plaining the remote control function. 
[0030] Fig. 17 is a schematic diagram used for ex- 
plaining a security function. 

[0031] Fig. 18 is a schematic diagram used for ex- 
20 plaining the security function. 

[0032] Fig. 19 is a schematic diagram used for ex- 
plaining an automatically maintenance and version-up 
function. 

[0033] Fig. 20 is a schematic diagram used for ex- 
25 plaining an E-mail management function. 

[0034] Fig. 21 is a schematic diagram used for ex- 
plaining a helper-requiring-person monitoring function. 
[0035] Fig. 22 is a schematic diagram used for a danc- 
ing function. 

30 [0036] Fig. 23 is a schematic diagram used for an in- 
ter-robot communicating function. 
[0037] Fig. 24 is a schematic diagram used for the in- 
ter-robot communicating function. 
[0038] Fig. 25 is a block diagram showing the struc- 

35 ture of a face-pattern studying and recognizing process- 
ing section. 

[0039] Fig. 26 is a block diagram showing the struc- 
ture of a synchronized-signal generator. 

40 Best Mode for Carrying Out the Invention 

[0040] An embodiment of the present invention will 
hereinafter be described with reference to the drawings. 

^5 (i) structure of robot according to this embodiment 

[0041] In Fig. 1, numeral 1 shows a robot according 
to this embodiment as a whole. Legs 5A to 5D compris- 
ing the upper halves of the legs 3A to 3D and the lower 

50 halves of the legs 4A to 4D are connected to the front- 
right, front-left, back-right and back-left sides of a body 
2, respectively. In addition, a tail 7 is connected to the 
back of the body 2, projecting upwards. 
[0042] In this case, as shown in Fig. 2, a head 6 is 

55 comprised of a microphone 1 0, a speaker 1 1 , image rec- 
ognizing section 1 2 being a CCD (Charge Coupled De- 
vice) camera, an image display 13 being a liquid display 
panel, a touch sensor 14 for detecting pressure given 
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by the outside and contact with the outside, a gas and 
smoke sensor for detecting leaking of gas and fire, an 
infrared transmitter/receiver 16 for transmitting and re- 
ceiving infrared signals 1R1, IR2, and an output-of-sen- 
sor receiver 17 being a radio receiver or the like. 
[0043] Further, the body 2 is comprised of a controller 

20 in the form of a microcomputer including a C PU (Cen- 
tral Processing Unit), a ROM (Read Only Memory), and 
a RAM (Random Access Memory), a sound processor 

21 having a synchronized-sound producingfunction and 
a sound recognizing function, an image processor 22 
having an image producing function, an acceleration 
and angular velocity sensor 23 for detecting the posture 
and position of the robot 1 , a key input section 24 being 
a keyboard having plural keys as shown in Fig. 1 for in- 
putting code numbers of requests desired by a user, and 
a communication unit 25 for making the controller 20 
communicate with the outside through a telephone line 
or a circuit for internet, or by radio or infrared signals. In 
addition, a battery 26 (Fig. 1 ) is attached to the stomach 
of the body 2 as a power source of this robot 1 . 
[0044] On the other hand, the tail 7 has an antenna 
27 electrically connected to the communication unit 25 
of the body 2. And, a plurality of power sensors 28A to 
28D for detecting pressure given from the outside are 
attached to the legs 5A to 5D, respectively. 

[0045] In addition, the joints in the legs 5A - 5D (the 
joints between the upper halves of the legs 3A - 3D and 
the lower halves of the legs 4A - 4D), the joints between 
each leg 5A - 5D and the body 2, the joint between the 
head 6 and the body 2, and a joint between the tail 7 
and the body 6 each has a joint mechanism constructed 
as described below. 

[0046] The microphone 10 of the head 6 gathers 
words given by the user and outside sounds like music 
and sounds, and transmits the obtained gathered-sound 
signal S1 to the sound processor 21. The sound proc- 
essor 21 recognizes the meanings of the words, which 
are gathered through the microphone 10, and of the 
words, which are transmitted from the outside through 
the telephone line, based on the gathered-sound signal 
S1 supplied from the microphone 10 and sound infor- 
mation supplied from the communication unit 25 as de- 
scribed later, and transmits the recognition result to the 
controller 20 and the communication unit 25 as a sound 
recognition signal S2A. Further, the sound processor 21 
generates a synchronized sound underthecontrol of the 
controller 20 and transmits this to the speaker 11 and 
the communication unit 25 as a sound signal S2B. 
[0047] The image recognizing section 12 photo- 
graphs the surroundings, and transmits the obtained 
photograph signal S3 to the image processor 22. The 
image processor 22 recognizes the outside situation 
photographed by the image recognizing section 12 and 
images transmitted through the telephone line or the like 
from the outside, based on the photograph signal S4A 
supplied from the image recognizing section 1 2 and the 
image information supplied from the communication unit 



25 as described below, and transmits the recognition re- 
sult to the controller 20 as an image recognition signal 
S4A. In addition, the image processor 22 performs pre- 
determined signal processing on the photograph signal 
5 S3 from the image recognizing section 1 2 underthe con- 
trol of the controller 20, and transmits the obtained im- 
age signal S4B to the communication unit 25. 
[0048] The touch sensor 14 detects physical works 
like "touching" and "patting" from the user and pressure 
w received when the head 6 contacts to an outside thing, 
and transmits the detection result to the controller 20 as 
a pressure detection signal S5. 

[0049] The gas and smoke sensor 15 always checks 
the concentration, of surrounding gas and the concen- 
tration of surrounding smoke, and when detecting gas 
or smoke of which concentration is more than a pre- 
scribed value, transmits to the controller 20 a gas and 
smoke detection signal S6 informing of this situation. 
[0050] The infrared transmitter/receiver 16 receives 
the infrared signal RI1 supplied from the user via the 
infrared remote controller as a control signal, and trans- 
mits the reception result to the controller 20 as an infra- 
red reception signal S7. 

[0051] The output-of-sensor receiver 17 receives an 
output of sensor transmitted various sensors by radio 
which are installed in a house for detecting the user's 
living condition, such as the is turned on in the kitchen, 
and whether gas is opened in the bathroom. Then, it 
transmits the reception result as an output-of-sensor re- 
ception signal S12 to the controller 20. 
[0052] And, the acceleration and angular velocity sen- 
sor 23 of the body 2 detects the acceleration and angular 
velocity of the time when the robot 1 changes its posture 
or when the robot 1 moves. Then it transmits the detec- 
tion result to the controller 20 as an acceleration and 
angular velocity detection signal S8. 
[0053] The key input section 24 recognizes the com- 
mand based on the code number inputted by the user, 
and transmits the recognition result to the controller 20 
as a key input signal S9. 

[0054] The communication unit 25, underthe control 
of the controller 20, communicates with the outside via 
the antenna 27 through the telephone line or a circuit for 
Internet, or by radio or infrared signals, to transmits the 
obtained image information to the image processor 22 
and also transmits sound information for voice, music 
or the like to the sound processor 21 . In addition, it trans- 
mits information such as commands and text data from 
the outside to the controller 20 as a received information 
signal S1 0. 

[0055] Further, the power sensors 28A - 28D of the 
legs 5A - 5D each detects the magnitude of power given 
by the user, and transmits the detection result to the con- 
troller 20 as a power detection signal S11. 
[0056] Based on the sound recognition signal S2A, 
the image recognition signal S4A, the pressure detec- 
tion signal S5, the gas and smoke detection signal S6, 
the infrared reception signal S7, the output-of-sensor re- 
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ception signal S12, the acceleration and angular veloc- 
ity detection signal S8, the key input signal S9, the re- 
ceived information signal S10 and the power detection 
signal S11 respectively supplied from the sound proc- 
essor 21, the image processor 22, the touch sensor 14, 
the gas and smoke sensor 15, the infrared transmitter/ 
receiver 1 6, the output-of-sensor receiver 1 7, the accel- 
eration and angular velocity sensor 23, the key input 
section 24, the communication unit 25, and the power 
sensors 28A - 28D, the controller 20 judges the sur- 
rounding situation , presence or absence of works by the 
user, the presence or absence of abnormality, the pres- 
ence or absence of commands from the user or the out- 
side, the user's living condition, and its own posture and 
position. 

[0057] And, the controller 20 decides its own next 
movement based on this judgement and the control pro- 
gram previously stored in an internal memory not 
shown, and makes the head 6 move upward and down- 
ward, makes the tail 7 wag back and fourth, makes the 
legs 5A to 5D move to walk, or the like by moving nec- 
essary joint mechanisms based on the decision. 
[0058] Further, the controller 20 controls the sound 
processor 21 , the image processor 22, or/and the infra- 
red receiver 16 as required, to give sounds based on 
the sound information received from the communicating 
party via the communication unit 25 and speaker 1 1 , to 
display images based on the image information re- 
ceived from the communicating party via the communi- 
cation unit 25 and images generated by the image proc- 
essor, on the image display 13, or to emit the infrared 
signal IR2 for controlling outside devices to the infrared 
receiver 16. 

[0059] In addition, the controller 20 controls the com- 
munication unit 25 as required, to transmit sound infor- 
mation based on the sound signal S2B from the sound 
processor 21 , the image information based on the image 
signal S4B from the image processor 22, and the com- 
mand information generated by the controller 20 itself 
in accordance with the manipulation of the outside, to 
the communicating party. 

[0060] In this way, the robot 1 , based on the its sur- 
rounding situations and the commands or the like from 
the user or the communicating party, can act in accord- 
ance with situations and commands. 
[0061] Note that, in the case of this embodiment, the 
communication unit 25 is composed of any of commu- 
nicating devices such as a telephone modem, a handy- 
phone of a cordless telephone, a mobile telephone, a 
transmitter/receiver for radio waves or a transmitter/re- 
ceiver for infrared signals, and is exchangeable together 
with the antenna 27. 

[0062] Therefore, this robot 1 selects a communica- 
tion state desired by the user from various kinds of com- 
munication states such as a telephone, an Internet, a 
radio signal, and an infrared signal , to communicate with 
outside in the selected communication state. 
[0063] Next, the concrete structures of the joint mech- 



anisms of the joints in the legs 5A - 5D, the joints be- 
tween each leg 5A - 5D and the body 2, the joint between 
the head 6 and the body 2. and the joint between the tail 
7 and the body 6 will be described in reference with Figs. 
5 3 and 4. 

[0064] As can be seen from Figs. 3 and 4, the joint 
mechanism between the body 2 and each leg 5A - 5D 
is composed of a bracket 30A - 30D provided on the 
body 2 as shown in Fig. 3, and two pairs of actuators 
10 34A : 34B, 34D, 34E, 34G, 34H, 34J, 34K each one hav- 
ing a motor 31A, 31B, 31D, 31E, 31G, 31 H, 31J, 31K, 
a position sensor 32A, 32B, 32D, 32E, 32G : 32H, 32J, 
32K : and a driver 33A, 33B, 33D, 33E, 33G, 33H, 33J, 
33K. 

15 [0065] The upper ends of the upper halves of the legs 
3A - 3D of the legs 5A - 5D are attached to the respective 
brackets 30 A - 30D in the joint mechanisms. The motors 
31 A : 31 D, 31 G, 31 J in one of the respective two actua- 
tors 34A, 34D, 34G, 34J are driven so as to rotate the 

20 upper halves of the legs 3A - 3D together with the brack- 
ets 31 A - 30D in the directions shown by arrows R1 - R4 
with central axes CL1 - CL4 as a center, respectively. 
On the other hand, the motors 31 B, 31 E, 31 H, 31 K in 
the other actuators 34B, 34E, 34H, 34K are driven so 

25 as to rotate the upper halves of the legs 3A - 3D in the 
directions shown by arrows R5 - R8 with central axes 
CL5 - CL8 as a center, respectively. 
[0066] Further, the joint mechanisms consisting the 
joints in the legs 5A - 5D (every joint between the upper 

30 half of the leg 3A - 3D and the lower half of the leg 4A - 
4D) each is composed of a connector 35A - 35D at- 
tached to the lower end of the upper half of the leg 3A - 
3D and an actuator 34C, 34F, 341 , 34L having a motor 
31 C, 31 F, 31 1, 31 L, a position sensor 32C, 32F, 321, 32L, 

35 and a driver 33C, 33F, 33I, 33L. 

[0067] In addition, the upper ends of the lower halves 
of the legs 4A - 4D are attached to the respective con- 
nectors 35A - 35D of the joint mechanisms. The motors 
31 C, 31 F, 31 1, 31 L of the corresponding actuators 34C, 

40 34F, 34I, 34L are driven to rotate the lower halves of the 
legs 4A - 4D in the directions shown by arrows R9 - R12 
with central axes CL9 - CL12 as a center. 
[0068] Further, the joint mechanism connecting the 
head 6 and the body 2 is composed of three pairs of 

45 actuators 34M, 34N, 34P each having a motor 31 M, 
31 N, 31 P, a position sensor 32M, 32N, 32P, and a driver 
33M, 33N, 33P. 

[0069] And, the head 6 is attached to the bracket in 
this joint mechanism. The motor 31 M of the first actuator 

so 34M is driven so as to make the head 6 shake in the 
direction of an arrow R1 3 with a central axis CL1 3 as a 
center. The motor 31 N of the second actuator 34N is 
rotated so as to make the head 6 shake in the direction 
of an arrow R14 with a central axis CL14 as a center. 

55 The third actuator 34P is driven so as to make the head 
6 shake in 

[0070] Furthermore, the joint mechanism connecting 
the body 2 and the tail 7 is composed of a bracket (not 
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shown) attached to the connecting position of tail 7 and 
the body 2, and an actuator 34Q comprising a motor 
31 Q, a position sensor 32Q, and a driver 33Q. 
[0071] Besides, the lower end of the tail 7 is attached 
to the bracket of this joint mechanism. The motor 31 Q 
of the actuator 34Q is driven so as to make the tail 7 
wag in the direction of an arrow R1 6 with the central axis 
CL16 as a center. 

[0072] In these joint mechanisms, each of the drivers 
33A - 33Q of the actuators 34A - 34Q is connected to 
the controller 20 with a bus 36 as shown in Fig. 4, and 
the motor 31 A - 31 Q and the position sensor 32A - 32Q 
are connected to the corresponding driver 33A - 33Q. 
[0073] Therefore, in this robot 1 , the motors 31 A - 31 Q 
desired by the controller 20 can be controlled through 
the corresponding drivers 33A - 34Q : and the rotary 
amounts of the motors 31 A - 31 Q which are detected by 
the corresponding position sensors 32A - 32Q at this 
time can be recognized through the drivers 33A - 34Q, 
thereby making it possible to rotate each motor 31 A - 
31 Q by a desired angle with accuracy. 
[0074] Note that, in this embodiment, the joint mech- 
anisms in the legs 5A - 5D (every joint between the up- 
per half of the leg 3A - 3D and the lower half of the leg 
4A - 4D) and the joint mechanisms between each of the 
legs 5A - 5D and the body 2 each has a brake mecha- 
nism 37. And using the function of this brake mecha- 
nism, this robot 1 can be directly taught a relative posi- 
tion relation of an arbitrary upper half of the leg 3A - 3D 
and lower half of the leg 4A - 4D in the legs 5A - 5D by 
a manipulator with the directly teaching method. 

(2) Each function of the Robot 1 

[0075] Next, each function of the robot 1 will be de- 
scribed. 

[0076] Note that, as described above, the communi- 
cation unit 25 of this robot 1 is exchangeable and can 
communicate with the outside in a communication state 
desired by a user. In the following explanation, as shown 
in Fig. 5 as one example, handyphones of a cordless 
telephone are applied as the communication unit 25 of 
the robot 1 in Mr. A's house 40 and the communication 
unit 25 of the robot 1 in Mr. B's house 41, and Mr. A's 
robot 1 and Mr. B's robot 1 can communicate through 
the telephone line via the telephones 42, 43, respective- 
ly. 

(2-1) Robot telephone function 

[0077] Firstly, this robot 1 has a robot telephone func- 
tion so that users can communicate with each other us- 
ing another one robot, which is the same as the robot 1 , 
with further reality as compared with communications 
on the phone. 

[0078] For example, when Mr. A manipulates the key 
input section 24 of Mr. A's robot 1 to input a prescribed 
first code number and the telephone number of Mr. B for 



making a phone call, the controller 20 of Mr. A's robot 1 
controls the communication unit 25 to call phone call 
from the outside via the communication unit 25, it con- 
trols the sound processor 21 to give sounds from the 

5 speaker 11 for informing of this phone call. 

[0079] Then, when the controller 20 of Mr. A's robot 1 
confirms that Mr. B answers this call by manipulating Mr. 
B's robot 1, it controls the communication unit 25 to 
transmit image information on an image of Mr. A photo- 

10 graphed by the image recognizing section 12, which is 
obtained based on the image signal S4B supplied from 
the image processor 22 to the communication unit 25 at 
this time, and sound information on Mr. A's voice gath- 
ered by the microphone, which is obtained based on the 

75 sound signal S2B supplied from the sound processor 21 
to the communication unit 25, to Mr. B's robot 1. 
[0080] At this time, the controller 20 of Mr. B's robot 1 
controls the image processor 22 to display the image of 
Mr. A based on the image information transmitted 

20 through the telephone line 44 as shown in Fig. 6, and 
on the other hand, controls the sound processor 21 to 
give Mr. A's voice based on the sound information trans- 
mitted through the telephone line 44 from the speaker 
11. 

25 [0081 ] In similar way, the controller 20 of Mr. B's robot 
1 controls the communication unit 25 to transmit image 
information on the image of Mr. B photographed by the 
image recognizing section 12, which is obtained based 
on the image signal S4B supplied from the image proc- 

30 essor 22 to the communication unit 25 at this time, and 
the sound information on Mr. B's voice gathered by the 
microphone 10, which is obtained based on the sound 
signal S2B supplied from the sound processor 21 to the 
communication unit 25, to Mr. A's robot 1. 

35 [0082] At this time, the controller 20 of Mr. A's robot 1 
controls the image processor 22 to display the image of 
Mr. B based on the image information transmitted 
through the telephone line 44 on the image display 13, 
and on the other hand, controls the sound processor 21 

40 to give Mr. B's voice based on the sound information 
transmitted through the telephone line 44 from the 
speaker. 

[0083] In this way, in this robot 1 , the users can com- 
municate with each other using two robots with sounds 

45 and images, like a video telephone. 

[0084] On the other hand, the controllers 20 of the ro- 
bots 1 of Mr. A and Mr. B each always checks sounds 
transmitted through the telephone line 44 from his com- 
municating party (Mr. B or Mr. A) based on the sound 

so recognition information S2A supplied from the sound 
processor 21.. 

[0085] When the controller 20 detects a previously 
registered word which indicates agreement or a re- 
sponse like "Yes" and "uh-huh M in the voice, it controls 
55 the corresponding joint mechanisms to move the head 
6 upward and downward for nodding as shown in Fig. 
7, for example. 

[0086] Further, when the controller 20 detects a pre- 
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viously registered word which indicates separation like 
"Bye-bye" and "See you", it controls the corresponding 
joint mechanisms to raise and shake one leg 5A for 
"Bye-bye" 

[0087] Furthermore, in similar way, when the control- 
ler 20 detects a word which indicates pessimism like 
"Sad° and "Sorry", when it detects a ward which indi- 
cates negative like "No way" and "No", and when it de- 
tects a word which indicates pleasure like "Got it!" and 
"happy", it slowly moves the head 6 downward for pes- 
simism as shown in Fig. 9 : shakes the head 6 for neg- 
ative as shown in Fig. 1 0, and wag the tail 7 for pleasure 
as shown in Fig. 11 , respectively, for example. 
[0088] Furthermore, other than these movements, 
when the controller 20 detects a ward, for example, 
"How big", it opens the front legs 5A, 5B of the robot 1 
to express "size", and moreover, when it detects voice 
of laughing, it makes the robot 1 perform movements as 
if the robot 1 shook its body with laughing. 
[0089] In this way, this robot 1 can act according to 
conversations based on voices of his communicating 
party, thereby the user can visually feel feelings of the 
communicating party. 

[0090] Note that, the communicating party can make 
this robot 1 perform these movements by remote con- 
trol. 

[0091] Actually, in the middle of conversation between 
Mr. A's robot 1 and Mr. B's robot 1 , when Mr. A manip- 
ulates the key input section 25 to input a prescribed sec- 
ond code number for manipulating the robot 1 of his 
communicating party (Mr. B), the controller 20 of Mr. A's 
robot 1 transmits this second code number to Mr. B's 
robot 1 via the communication unit 25. 
[0092] As a result, the controller 20 of Mr. B's robot 1 
controls the corresponding joint mechanisms based on 
the supplied second code number and a previously 
memorized table for movement patterns with respect to 
the second code numbers, in order to perform the move- 
ments shown in Figs. 7 - 11 , movements showing size, 
or movements showing laughing. 

[0093] In addition, in these robots 1 , two robots 1 be- 
ing communicating can perform the same movement. 
[0094] In actual, while Mr. A's robot 1 and Mr. B's robot 
1 are communicating with each other, for example, when 
Mr. A manipulates the key input section 24 to input a 
third code number for moving his own robot 1, the con- 
trolled of Mr. A's robot 1 drives the corresponding joint 
mechanisms based on a previously memorized table for 
movement patterns with respect to the third code num- 
bers, to perform the designated movements. 
[0095] In addition to this, the controller 20 of Mr. A's 
robot 1 transmits this third code number to Mr. B's robot 
1 via the communication unit 25. 

[0096] At this time, the controller 20 of Mr. B's robot 1 
drives the corresponding joint mechanisms based on 
the supplied third code number and a previously mem- 
orized table for movement patterns with respect to the 
third code numbers, to perform the designated move- 



ments. 

[0097] Further, in these robots 1 , a power from the 
outside (pressure of shaking hands or the like) which 
one of two robots 1 being communicating with each oth- 

5 er gets can be communicated through the other robot 1 . 
[0098] In actual, in these robots 1, while Mr. A's robot 
1 and Mr. B's robot 1 are communicating with each other, 
the controller 20 of Mr. A detects the given pressure 
based on the power detection signal S1 1 supplied from 

10 the power sensors 28A - 28B, and transmits the pres- 
sure detection result to Mr. B's robot 1 via the commu- 
nication unit 25. 

[0099] At this time, the controller 20 of Mr. B's robot 1 
drives the corresponding joint mechanisms based on 
is the supplied pressure detection result to generate the 
pressure, which Mr. A got, in the corresponding legs 5A 
-5D. 

[0100] In this way, in this robot 1 , the intentions, feel- 
ings and movements of the user can be transmitted to 
20 his communicating party as the movements of the robot 
1. 

(2-2) Answering telephone function 

25 [0101] On the other hand, this robot 1 also has an an- 
swering telephone function. 

[01 02] In actual, when the controller 20 of the robot 1 
knows a phone call from the outside via the communi- 
cation unit 25, it controls the sound processor 21 as de- 
30 scribed above to give sounds from the speaker 11 for 
informing of this phone call. 

[0103] Then, when nobody answers this phone call 
within a 21 to produce a fixed message (synchronized 
voice) for the answering telephone, and transmits this 

35 to the communication unit 25 as a sound signal S2B. 
[01 04] At this time, for example, the controller 20 judg- 
es the user's condition of this time as shown in Fig. 13, 
based on outputs of various sensors installed in the 
house as shown in Fig. 12, which are obtained based 

40 on the output-of-sensor reception signal S12 from the 
output-of-sensor receiver 17 (Fig. 2), and the current 
time information supplied from a clock not shown. 
[0105] Then, the controller 20 makes the sound proc- 
essor 21 produce an optimal message based on this 

45 judgement as a message for the above answering tele- 
phone and transmit this to the communicating party via 
the communication unit 25. 

[0106] After that, the controller 20 stores sound infor- 
mation and/or image information transferred from a 
50 communicating party into a memory not shown for mem- 
ory until a predetermined time is passed or the line is 
. cut off. 

[01 07] Then, when a request for reproducing the mes- 
sage is inputted by manipulating the key input section 
55 24 or the like, the controller 20 gives the voice based on 
the memorized sound information from the speaker 11 
and also displays the image based on the memorized 
image information on the image display 13. 
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[01 08] In this way, in this robot 1 , as an answering tel- 
ephone, messages according to user's conditions are 
transmitted to the communicating party, and also mes- 
sages and images from a communicating party are 
memorized. 

[0109] Furthermore, when a call waiting mode is se- 
lected by manipulating the key input section 24 in the 
middle of conversation, the controller 20 of the robot 1 
controls the sound processor 21 to produce predeter- 
mined music and/or sounds, and transmits this to the 
communicating party via the communication unit 25. In 
this way, this robot 1 can give music and/or a message 
to a communicating party for a call waiting. 

(2-3) Call connecting function 

[0110] On the other hand, as shown in Fig. 12, this 
robot 1 has a call connecting function for connecting a 
phone call from the outside to a corresponding user of 
plural users (Mr. A, Mr. B, and Mr. C). 
[0111] In actual, when the controller 20 of the robot 1 
knows a phone call from the outside via the communi- 
cation unit 25, it detects the telephone number of this 
phone call, for example, by using a number display serv- 
ice which is usually utilized in telephones. 
[0112] Then, the controller 20, based on thus detect- 
ed telephone number of that phone call and the plural 
telephone numbers previously registered correspond- 
ing to Mr. A, Mr. B and Mr. C, judges who out of Mr. A, 
Mr. B, and Mr. C that phone call is for. 
[0113] At this time, if the telephone number of that 
phone call can not be detected or if the telephone 
number of that phone call has not been registered, the 
controller 20 controls the sound processor 21 to produce 
a message for asking who that phone call is for, and 
transmits this sound information to the communicating 
party via the communication unit 25 . Then, the controller 
20, based on the sound information which is given from 
the communicating party as a response, judges based 
on the sound recognition signal S2A from the sound 
processor 21 who out of Mr. A, Mr. B and Mr. C that cal 
is for. 

[0114] Then, if the controller 20 judges that that phone 
call is for Mr. A, it looks for Mr. A while moving by con- 
trolling the joint mechanisms of the legs 5A - 5D as re- 
quired, based on the image recognition signal S4A sup- 
plied via the image processor 22 and the previously reg- 
istered image information on Mr. A's face. 
[0115] When the controller 20 finds Mr. A through this 
detection, it controls the joint mechanisms of the legs 
5 A - 5D so as to move the robot 1 toward Mr. A. 
[0116] Then, when the robot 1 comes nearer to Mr. A 
within a predetermined distance, the controller 20 con- 
trols the corresponding joint mechanisms to produce 
movements, for example, "sorrow" as described in Fig. 
9 in the case where that phone call for Mr. A is from a 
person who is previously registered as "a unfavorite"; 
and in the case where the phone call is from a person 



who is previously registered as "a favorite", for example, 
movements "pleasure" are performed as described 
above in Fig. 11. 

[0117] In this way, in addition to the call connecting 
5 function, this robot can inform a user of a person, who 
is calling, with movements. 

(2-4) Remote control function 

10 [0118] In addition, this robot 1 has a remote control 
function which enables the user to call the robot 1 with 
an outside device 50 such as an external telephone and 
mobile phone, to use the dial buttons of the outside de- 
vice 50 in order to input a predetermined fourth code 

is number, and to make the robot 1 turn ON/OFF the 
switches of a tape recorder 51 , a television set 52 and 
the like or turn ON/OFF the switches 53, 54 on the wall 
or on the floor. 

[0119] In actual, when the controller 20 of the robot 1 

20 knows a phone call from the outside device 50 via the 
communication unit 25 and then confirms the input of 
the fourth code number, it recognizes the request based 
on this number and the previously memorized table of 
movements with respect to the fourth code numbers, 

25 and executes the predetermined processing previously 
set, based on the recognition result. 
[0120] For example, in the case of a request to turn 
ON/OFF an electrical appliance such as the video tape 
recorder 51 and the television set 52, which can be con- 

30 trolled under remote control, the controller 20 controls 
the joint mechanisms of the legs 5A - 5D while confirm- 
ing the surrounding situations based on the image rec- 
ognition signal S4A from the image processor 22, to 
move the robot 1 to the setting place of the electrical 

35 appliance. 

[0121] Then, when the robot 1 moves up to the area 
where the electrical appliance can be controlled under 
remote control, the controller 20 controls the infrared 
transmitter/receiver 15 to emit the infrared signal RI12 

40 as a control signal toward the electrical appliance, there- 
by turning ON/OFF the power of the electrical applianc- 
es. 

[0122] Further, in the case of a request to turn the 
switch 53, 54 on the wall or on the floor ON/OFF, the 

^5 controller 20 controls the joint mechanisms of the legs 
5A - 5D to move the robot 1 to the previously memorized 
setting place of the switch 53, 54 while confirming the 
surrounding situations based on the image recognition 
signal S4A from the image processor 22. 

so [0123] Then, when the robot 1 moves up to the area 
where the switch 53, 54 can be controlled, the controller 
20, in the case where the switch 53 is on the wall, pushes 
the touch sensor 14, which is attached to the head 7, 
against the switch 53 to turn the switch 53 ON/OFF. 

55 [01 24] At this time, the controller 20 recognizes based 
on the output of the touch sensor 14 whether the touch 
sensor 14 touched the switch 53, thereby it can surely 
execute ON/OFF operation. 
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[0125] On the other hand, in the case where the 
switch 54 is on the floor, the controller 20 controls the 
joint mechanisms of the corresponding leg 5A - 5D to 
push the leg 5A - 5D against the switch 54, thereby the 
switch 54 is turned ON/OFF. 

[01 26] Fig. 1 6 shows examples of other works of the 
robot 1 . The robot 1 can turn ON/OFF the switch 56 of 
the hot-water supply switch 58 for automatically control- 
ling a blind 57 to move a motor 59 so as to open/close 
the blind 57. 

[0127] In addition, the robot 1 turns the switch 61 of 
the washing-taking device 60 ON, so that the washing- 
taking device 60 is driven to take washings 62 into the 
house 63 from the outside. 

[0128] Further, in the case where movements desig- 
nated under remote control are performed, if the con- 
troller 20 recognizes, based on the image recognition 
signal S4A from the image processor or the sound rec- 
ognition signal S2A from the sound processor 21, that 
the controlled device was not be correctly driven by a 
switch operation, it controls the sound processor 21 and 
the communication unit 25 to produce a message of in- 
forming of this error and transmits this to the user. 
[0129] In this way, using this robot 1 , the user at a re- 
mote place can confirm whether the robot can surely fin- 
ish operations designated by him or not, and can check 
the reason for this error by controlling the robot 1 as re- 
quired. 

(2-5) Security function 

[0130] In addition, this robot has a security function to 
detect trouble. 

[0131] In actual, the controller 20 of the robot 1 always 
checks input of the gas and smoke detection signal S6 
from the gas sensor and smoke sensor 1 5 (Fig. 2), and 
if receiving the gas and smoke detection signal S6, it 
controls the sound processor 21 to give sounds from the 
speaker 11 to inform of this situation. 
[0132] In addition to this, in the case where an inform- 
ing mode is set, the controller 20 controls the commu- 
nication unit 25 and the sound processor 21 to transmit 
a prescribed message to the previously registered us- 
er's contact number and moreover, after that, if a pre- 
determined time is passed in this situation or if a large 
amount of gas or smoke is detected, the controller 20 
controls the communication unit 25 and the sound proc- 
essor 21 and calls the police or the fire department to 
inform of this situation, as shown in Fig. 17. 
[01 33] Further, for example, in the case where an an- 
swering telephone mode of the security function is set, 
the controller 20 always checks the house's condition 
based on the image recognition signal S4A from the im- 
age processor 22 and the sound recognition signal S2B 
from the sound processor 21 . If a person who can not 
be recognized or identified (who is not previously regis- 
tered) is detected, the controller 20 controls the commu- 
nication unit 25 and the sound processor 21 to transmit 



a prescribed message to the previously registered us- 
er's contact number and the police, as shown in Fig. 18. 
[0134] Furthermore, in addition to these functions, in 
the case where the gas and smoke detection signal S6 

5 is input or in the case where a person who can not be 
recognized or identified is detected while the answering 
telephone mode of the security function is set ON, the 
controller 20 sequentially memorizes image the sound 
processor 21 at this time, and then, when a reproduction 

10 request is made, it displays images based on the mem- 
orized image information on the image display 13 and 
gives sound based on the memorized sound information 
from the speaker 11. 

[0135] In this way, the robot 1 can memorize and show 
15 situations in the case of disaster, accident, illegal intrud- 
er, or the like, like a voice recorder of an airplane. 

(2-6) Automatically maintenance and version-up 
function 

20 

[0136] In addition, this robot 1 has an automatically 
maintenance and version-up function for automatically 
performing maintenance by calling a maintenance cent- 
er 60 or for automatically updating a version. 

25 [0137] In actual, the controller 20 of the robot 1 regu- 
larly calls the maintenance center 60 by controlling the 
communication unit 25. And the controller 20 communi- 
cates with a check device of the maintenance center 60 
to check variousf unctions including the controller 20 un- 

30 der the control of the check device. If the controller 20 
detects abnormality in any parts, it communicates this 
situation to the check device of the maintenance center 
60, and moreover, controls the image processor 22 to 
display a message on the image display 1 3 for informing 

35 of this situation. 

[01 38] Further, the controller 20 controls the commu- 
nication unit 25 to regularly call the maintenance center 
60. Then, the controller 20 takes in an updated program 
supplied from the maintenance center 60 via the com- 

^0 munication unit 25, and stores it in an internal memory 
for programs, not shown, in place of the old program. 
[0139] Further, with respect to the updated programs 
which are one-sidedly given through the telephone line, 
the controller 20 takes in these via the communication 

45 unit 25 similarly to the above cases, and then stores 
these in a memory for programs as described above. 
[0140] In this way, this robot 1 can automatically per- 
form maintenance or update of a version. 

so (2-7) E-mail management function 

[0141] On the other hand, this robot 1 has an E-mail 
management function for managing E-mail. 
[0142] In actual, when the controller 20 of the robot 1 
55 recognizes the reception of E-mail, it takes in the text 
data in an internal memory not shown. 
[0143] Next, the controller 20 judges based on data 
about a designation who this E-mail is for, and then it 
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looks for the corresponding user with the method de- 
scribed in the call connecting function, to come nearer 
to him, based on this judgement. After that, it controls 
the sound processor 21 to give a message from the 
speaker 11 for informing of this E-mail. 
[0144] At this time, similarly to the case described in 
the call connecting function, for example, if the sender 
of E-mail is previously registered as "an unfavorite", the 
controller 20 for "sorrow" as described in Fig. 9; and in 
the case where the sender of E-mail is previously reg- 
istered as "a favorite", the movement for "pleasure" as 
described in Fig. 11 is produced. 
[0145] In this way, this robot 1 can inform the user of 
the sender of E-mail with movements, in addition to the 
management. 

(2-8) Helper-requiring-person monitoring function 

[0146] Further, this robot 1 has a helper-requiring- 
person monitoring function for monitoring a user, such 
as an elderly person and a child, who needs a helper or 
monitor. 

[0147] In actual, in the helper-requiring-person mon- 
itoring function is set ON, the controller 20 of this robot 
1 always monitors the user's movements and conditions 
designated by another user based on the image recog- 
nition signal S2A from the image processor 22, the 
sound recognition signal S4A from the sound processor 
21 , and the output-of-sensor reception signal S12 from 
the output-of-sensor receiver 1 7 (Fig. 2). 
[0148] If the user does not move over a predeter- 
mined time or if the user does not give voice, the con- 
troller 20 looks for the user to come nearer to him as 
described in the call connecting function. 
[01 49] Then, when the controller 20 comes nearer to ' 
the user up to a predetermined distance, it controls the 
sound processor 21 to give a prescribed message or 
music from the speaker 11 . After that, it keeps on mon- 
itoring the user's movements, and if he shows no reac- 
tion, it controls the sound processor 21 and the commu- 
nication unit 25 to call the previously registered family 
and hospital, and transmits a message to inform of this 
trouble. 

(2-9) Dancing function 

[0150] Further, this robot 1 has a dancing function of 
dancing to music gathered by the microphone 10 or of 
dancing based on music transmitted through the tele- 
phone line 44 or MIDI (Musical Instrument Digital Inter- 
face) signals generated by coding note information. 
[0151] Here, the MIDI standard is the international 
standard for connecting a synthesizer, a rhythm box, a 
sequencer, a computer, and so on, to each other. In ad- 
dition, the MIDI standard is defined, in which information 
such as note-on (give sounds), note-off (stop sounds), 
pitch-bender (continuously change scales) and control 
change (other various kinds of control) is transmitted to- 



gether with an identification number as a MIDI signal at 
130 per one second. As a result, all information ex- 
pressed by playing the musical instruments by people 
can be transmitted as MIDI signals. 

5 [0152] In the case where the dancing function is set 
ON : when the MIDI signal through the telephone line 44 
is supplied, the sound processor 21 of this robot 1 takes 
in this via the communication unit 25, extracts informa- 
tion on music characteristics such as rhythm, intervals, 

10 strength of tone, from the MIDI signal and moreover, 
produces a synchronized signal based on the extracted 
information. 

[0153] For example, the sound processor 21 extracts 
only information on a dram, and with timing of beating 
*5 the drum as a reference on the basis of the extracted 
information, produces a synchronized signal by gener- 
ating pulse at that timing, and then transmit this to the 
controller 20. 

[0154] Further, in the case where music is gathered 
20 by the microphone or in the case where the music (non- 
MIDI signals) is supplied through the telephone line or 
the like, the sound processor 21 detects music charac- 
teristics such as rhythm, intervals, strength of tone of 
the music, based on the music obtained based on the 
25 gathered signal 1 given from the microphone 1 0 or the 
music transmitted through the telephone line, and more- 
over, produces a synchronized signal based on the mu- 
sic characteristic of the music based on the detection 
result similar to the above case, and transmits this to the 
30 controller 20. 

[01 55] Then, the controller 20 moves the joints mech- 
anisms of the joint parts in the legs 5A - 5D, the con- 
necting parts between each leg 5A - 5D and the body 2 
and the connecting part between the head 6 and the 
35 body 2 based on thus generated synchronized signal, 
so that the robot 1 moves as if it danced to rhythm, as 
shown in Fig. 22. 

(2-10) Inter-robot communicating function 

40 

[0156] Furthermore, this robot 1 has an inter-robot 
communicating function, so that the robots 1 can play 
such a soccer game together while communicating. 
[0157] In actual, in the case where the inter-robot 

45 communicating function is set ON, the controller 20 of 
the robot (hereinafter, referred to as the first robot) 1 
monitors the position and condition of the other robot 
(hereinafter, referred to as the second robot) 1 based 
on the image signal S4 from the image processor 22. 

50 [0158] When the controller 20 of the first robot 1 wants 
the second robot 1 perform desired movements, ittrans- 
. mits to the second robot 1 a request to perform the 
movements via the communication unit 25. 
[01 59] At this time, the controller 20 of the second ro- 

55 bot 1 drives the required joint mechanisms based on the 
movement request supplied via its own communication 
unit 25, to perform the designated movements. 
[0160] In this way, in these robots 1 , the robots 1 con- 
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trol each other's robots 1 to work together while com- 
municating with each other. 

[0161] Note that, Fig. 24 shows the structure example 
of the game system 70, for example, in which two users 
play the soccer game by using such the inter-robot com- 
municating function. 

[0162] In the game system 1 , two personal computers 
71 A, 71 B are used and each user uses his own personal 
computer 71 A, 71 B to input a performance request such 
as "kick the ball forward and attack" and "attack from 
the right and shoot the ball". 

[0163] In this case, these personal computers 71 A, 
71 B transmit to the robots 1 of respective teams a com- 
mand corresponding to the input request, through the 
Internet 72, a communication server73, a telephone line 
74, and the telephone 75A, 75B in order. 
[0164] As a result, the robots 1 of the respective 
teams execute the requests while communicating with 
each other and control each other's robots 1 based on 
requests, as described above. 

[01 65] In this way, in this game system 70, two users 
control the robots of the respective teams to play the 
soccer game. 

(3) User identifying method of robot 1 

[0166] Next, a human recognizing and identifying 
method for realizing the above-mentioned call connect- 
ing function, helper-requiring-person monitoring func- 
tion, security function and so on, will be described. 
[01 67] At first, in this robot 1 , the image processor 22 
recognizes and identifies users, for example, by using 
the method disclosed in Japan Patent Laid Open No. 
89344/94. Concretely, a face pattern studying and rec- 
ognizing part 80 as shown in Fig. 25 is provided in the 
image processor 22 to recognize and identify users. 
[01 68] This face pattern studying and recognizing pat 
80 quantizes the face part of a human from image infor- 
mation which is supplied from the image recognizing 
section 12, with a memory 81 composed of a RAM (Ran- 
dom Access Memory) and an analog/digital converter, 
for example, every eight bits, and stores the obtained 
face image data l(x,y) being secondary luminance infor- 
mation on the x-y plane, in the RAM of the memory 81 . 
[0169] A preprocessor 82 calculates, as to each of r 
pieces of functions Fj(x.y) (i=1 ,2,...,r) previously stored 
in a function studying and storage section 84 as a basic 
model of the face image data l(x,y) stored in the memory 
81, a contribution degree Xj which is the conelation 
amount with respect to the face pattern P(x,y). 
[0170] Further, a comparison processor 83 detects a 
function F MAX (x,y) having the maximum contribution de- 
gree X MAX (1^MAX^r) based on the calculation result, 
changes the function F MAX (x,y) or the face pattern P(x, 
y) until the contribution degree X MAX of this function 
F MAX (x,y) becomes the maximum or the largest point, 
in order to calculate the change amount M(x,y) which is 
the difference between the function F MAX (x,y) and the 



face pattern P(x,y). 

[0171] This change amount M(x,y) is supplied to the 
function studying and storage section 84 and a change 
amount analyzing section 85 . In addition, to the function 
5 studying and storage section 84, the face pattern P(x s y) 
is also supplied. 

[0172] The function studying and storage section 84 
is formed of the neural-network, and stores the r pieces 
of functions Fj(x,y) (i=1 ,2,...,r) as a basic model of the 

10 face pattern P(x,y) as described above. 

[0173] Then, the function studying and storage sec- 
tion 84 uses the supplied change amount M(x,y) to 
change the function F MAX (x,y) or the face pattern P(x, 
y), and updates the function F MAX (x,y) based on the 

15 changed function F MAX f (x : y) and the changed face pat- 
tern P'(x,y) on the x-y plane. 

[01 74] On the other hand, the change amount analyz- 
ing section 85 analyzes the change amount M(x,y) sup- 
plied from the comparison processor 83, eliminates 

20 components of the differences in top, bottom, right, and 
left sides, in the shift due to rotation, or of the difference 
in size due to such the perspective and the expansion 
and reduction of the image recognizing section 12 (Fig. 
2), of the face pattern P(x,y) on the screen, from the 

25 change amount M(x,y), in order to generate a new 
change amount Mtdr(x.y), and then transmits this to a 
human information studying and storage section 86. 
[0175] In the case where the movement mode is a 
study mode, the human information studying and stor- 

30 age section 86 stores the supplied change amount Mtdr 
(x,y) into an internal memory not shown by correlating 
it with human information K(t) which is the function of 
numbers t (t=1 ,2 : ...,T; T is the number of human's faces) 
assigned to, for example, humans (faces) and is sup- 

35 plied from the controller 20 (for example, treating the 
average of plural change amounts Mtdr(x,y), Mtdr'(x,y), 
Mtdr"(x,y)... with respect to the face image of the same 
person t as the human information K(t)). 
[01 76] That is, in the case where the movement mode 

40 is the study mode, the human information studying and 
storage section 86 stores the change amount Mtdr(x,y) 
itself of a human t outputted from the change amount 
analyzing section 85 as human information, and after 
that, whenever the change amount Mtdr(x,y) ot the 

^5 same person t is input, it updates the human information 
K(t) based on the change amount Mtdr(x.y). 
[0177] Further, in the case where the movement 
mode is the recognizing mode, the human information 
studying and storage section 86 calculates, for example, 

50 an Euclidean distance between the change amount Mt- 
dr.(x.y) supplied from the change amount analyzing sec- 
tion 85 and the human information K(t) previously stored 
in the internal memory, and transmits the number t of 
the human information K(t) having the shortest Euclide- 

55 an distance as a recognition result KO to the controller 
20. 

[0178] Then, the controller 20 recognizes who the 
photographed person is (the person's name), based on 
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this recognition result KO and the person's name previ- 
ously stored corresponding to the number t at the time 
of registration. 

[0179] Then, in the face pattern studying and recog- 
nizing part 80 having such the structure, the change 
amount analyzing section 85 analyzes the change 
amount M(x,y), eliminates the parallel movement com- 
ponents, the rotary movement component, and the ex- 
pansion and reduction component, of the face pattern 
P(x,y), which are included in the change amount M(x,y), 
and updates the reference pattern stored in the human 
information studying and storage section 86 based on 
the new change amount Mtdr(x,y). As a result, the high 
recognition rate can be obtained. 

(4) Synchronized signal producing method of robot 1 

[0180] Sequentially, a method of producing the afore- 
mentioned synchronized signal for realizing the afore- 
mentioned dancing function will be described. 
[01 81 ] In this robot 1 , a synchronized signal producing 
part 90 as shown in Fig. 26 is provided in the sound proc- 
essor 21, as a method of making the robot 1 dance 
based on the music gathered by the microphone 1 0 : the 
music characteristics of music transmitted through the 
telephone line, and the music characteristics included 
in MIDI signals supplied through the telephone line 44 
from the outside. 

[0182] The synchronized signal producing part 90 in- 
puts a MIDI signal S20 supplied from the outside via the 
communication unit 25 to a MIDI signal processor 91. 
The MIDI signal processor 91 extracts the music char- 
acteristics such as the rhythm, the interval and the 
strength of sounds, which are previously determined, 
from the supplied MIDI signal S20 and moreover, pro- 
duces the synchronized signal S21 based on the ex- 
tracted music characteristics, and outputs this to the 
controller 20. 

[01 83] The synchronized signal processing part 90 in- 
puts the gathered sound signal S1 supplied from the mi- 
crophone 10 and the music information supplied from 
the communication unit 25 to a non-MIDI signal proces- 
sor 91 . In this case, the non-MIDI signal processor 91 
selects a signal processing mode from three signal 
processing modes by the outside manipulation. 
[0184] When the first signal processing mode is se- 
lected, for example, the gathered signal S1 from the mi- 
crophone 1 0 or the music information from the commu- 
nication unit 25 is supplied to a band pass filter 93 which 
is composed of a variable band pass filter circuit or plural 
band pass filter circuits. The band pass filter 93 extracts 
the component of signals having a predetermined fre- 
quency bandwidth, which is included in the gathered sig- 
nal S1 or the music information. As a result, the signal 
component, for example, of a predetermined musical in- 
strument (for example, a drum) continuously played is 
extracted and supplied to a power measuring section 94 
as a filter processed signal S22. 



[0185] The power measuring section 94 detects pow- 
er (strength) of sounds based on the supplied filter proc- 
essed signal every predetermined interval (for example, 
every 10 [ms]), and sequentially transmits the detection 
5 results to a characteristic controller 95 as a power de- 
tection signal S24. 

[0186] The characteristic controller 95 detects a part 
(time) exceeding a predetermined set threshold value 
from the supplied power detection signal S24. In addi- 

10 tion, the characteristic controller 95 measures the 
rhythm, the intervals and the strength of sounds on a 
time axis based on the detection result, and produces 
the aforementioned synchronized signal S2t based on 
the measurement result. 

is [01 87] Then, in such the first signal processing mode, 
the synchronized signal S21 is produced to make a ro- 
bot dance to the tempo and power, mainly of like a drum, 
and supplied to the controller 20. 

[0188] On the other hand, in the case where the sec- 
20 ond signal processing mode is selected, the non-MIDI 
signal processor 92 supplies the gathered signal S2 
from the microphone 10 or the music information from 
the communication unit 25 to the band pass filter 93, 
and the gathered signal S1 or the music information is 
25 subjected to filtering processing for all bandwidth there, 
and then, the obtained filtering processed signal S23 is 
supplied to a basic frequency extracting section 95. 
[0189] The basic frequency extracting section 95 ex- 
tracts a basic frequency from the supplied filtering proc- 
30 essed signal S23, and the obtained basis frequency sig- 
nal S25 is supplied to the characteristic controller 96. 
The characteristic controller 96 measures the supplied 
basis frequency signal S25 as the difference on the time 
axis, and produces the aforementioned synchronized 
35 signal S21 based on the specific different information. 
[0190] Then, in such the second signal processing 
mode, such the synchronized signal S21 as to make a 
robot dance to melody of music played with plural music 
instruments is produced and supplied to the controller 
40 20. 

[0191] Otherwise, in the case where the third signal 
processing mode is selected, in the non-MIDI signal 
processor 92, the gathered signal S1 from the micro- 
phone 10 or the music information from the communi- 
45 cation unit 25 is supplied to the band pass filter 93 : and 
the component of signals having the specific frequency 
bandwidth, which is included in the gathered signal 1 or 
the music information, is extracted there similarly to the 
first signal processing mode. Then, the obtained filtering 
50 processed signal S23 is supplied to the power measur- 
ing section 94. As a result, similarly to the first signal 
processing mode, the power detection signal S24 based 
on the filtering processed signal S23 is generated at the 
power measuring section 94 and supplied to the char- 
55 acteristic coniroller 96. 

[0192] In addition to this, the band pass filter 93 ap- 
plies the filtering processing to all bandwidth of the gath- 
ered signal S1 or the music information, and transmits 



25 



30 



35 



40 



45 



12 



NSDOClD:<EP 1122U38A1 ! > 



23 



EP 1 122 038 A1 



24 



the obtained filtering processed signal S23 to the basic 
frequency extracting section 95, similarly to the second 
signal processing mode. As a result, similarly to the sec- 
ond signal processing mode, the basic frequency signal 
S25 based on the filtering processed signal S23 is pro- 
duced at the basic frequency extracting section 95, and 
then supplied to the characteristic controller 96. 
[0193] Then, the characteristic controller 96, based 
on the supplied power detection signal S24 and basic 
frequency signal S25, produces the synchronized signal 
S21 with the combination of processing in the first and 
second signal processing modes. 
[01 94] In actual, in the case where a drum which has 
few musical interval difference is the main musical in- 
strument even the same music, the characteristic con- 
troller 96 produces the synchronized signal S21 based 
on the tempo and the strength of sound, similarly to the 
first signal processing mode. Further, in the case where 
a stringed instrument which gives silence sounds and 
has less difference of strength of sounds is the main mu- 
sical instrument, the characteristic controller 96 produc- 
es the synchronized signal S21 based on the difference 
of the basis frequency, similarly to the second signal 
processing mode. Furthermore, in the case of the mix- 
ture of these, it performs a mutual correction with at least 
two characteristics to produce the synchronized signal 
S21. 

[0195] In such the third signal processing mode, the 
synchronized signal S21 is produced with high accuracy 
by the mixture of good processing in thefirst and second 
processing mode and is supplied to the controller 20. 
[01 96] In this way, the synchronized signal producing 
part 90 produces the synchronized signal S21 . 

(5) Operation and Effect of this embodiment 

[0197] In the aforementioned configuration, this robot 
1 displays images based on image information on the 
image display 13 and gives sounds based on sound in- 
formation from the speaker 11, which are supplied 
through the telephone line 44 from the outside, and at 
the same time, moves in accordance with conversa- 
tions. Further, the robot 1 performs designated move- 
ments based on a movement request (a second or third 
code number) supplied from the outside through the tel- 
ephone line 44. 

[01 98] Therefore, using this robot 1 enables the trans- 
mission of feelings via sounds and images and further, 
the transmission of feelings via the movements of the 
real robot 1 , thereby making it possible to communica- 
tion with reality as if a communicating party was in front 
of you. In addition, for example, by using this robot 1 , it 
is possible for a family and a doctor to call the robot 1 
shown in Fig. 1 to monitor children and elderly people 
via the image recognizing section 12 of the robot 1 and 
to smoothly communicate by talking via the robot 1 when 
the children and elderly people who need helpers are in 
a house. 



[0199] Further if nobody answers a phone call from 
the outside within a predetermined time, the robot 1 
judges the user's condition based on the outputs of var- 
ious sensors disposed in the house, and transmits a 
5 message based on the judgement to a communicating 
party. Therefore, by using this robot 1, it is possible to 
give a suitable message for an answering telephone to 
the communicating party without troublesome setting 
work. 

10 [0200] Furthermore, the robot 1 recognizes which us- 
er a phone call is for, by a number display service or 
asking with sounds when receiving the phone call from 
the outside, and comes nearer to the user based on the 
recognition. Thus, using this robot 1 enables a user to 

75 talk to a communicating party on the phone even if the 
user is too busy to go and get a receiver. 
[0201] Furthermore, this robot 1 turns ON/OFF the 
switches of the video tape recorder 51 and the television 
set 52, and the switches 53, 54 on the wail and floor in 

20 the house, based on a request given from the user 
through the telephone line 44. Therefore, by using this 
robot 1 , it is possible to control electrical appliances and 
switches which can not be manipulated by the telephone 
operation. 

25 [0202] Furthermore, this robot 1 always recognizes 
the condition of a house, and calls a user, police, fire 
department orthe like if discovering abnormality. By us- 
ing this robot 1 , it is possible to secure security of a 
house. 

30 [0203] Furthermore, this robot 1 regularly calls the 
maintenance center 60 to check its various functions or 
to download the latest programs. Therefore, this robot 
1 enables a user to use it without considering its condi- 
tion. 

35 [0204] Furthermore, this robot 1 detects a user being 
the designation and comes nearer to him when receiv- 
ing E-mail. Therefore, using the robot 1 enables the user 
to recognize the E-mail with ease. 
[0205] Furthermore, this robot 1 always checks a des- 

40 jgnated user and calls another user or a hospital previ- 
ously registered if discovering abnormality. Therefore, 
by using the robot 1 , it is possible to handle with the case 
where a person who needs a helper is into trouble. 
[0206] Furthermore, this robot 1 dance to rhythm 

45 based on music formed of sounds gathered from the 
outside and the M IDI signal S20 transmitted through the 
telephone line 44. Therefore, this robot 1 can improve 
its amusement. 

[0207] Furthermore, this robot 1 controls another ro- 
50 bot 1 while they communicate with each other. There- 
fore, this robot 1 can makes plural robots 1 work togeth- 
er. 

[0208] According to the aforementioned configura- 
tion, the robot 1 displays and gives images and sounds 
55 based on image information and sound information sup- 
plied from the outside through the telephone line 44, 
moves in accordance with conversions, and performs 
designated movements based on a movement request 
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given from the outside through the telephone line 44, 
thereby making it possible to transmit feelings via 
sounds and images and feelings via movements of the 
real robot 1 so as to communicate with reality. Thus, a 
robot with high usability can be realized. 5 
[0209] In addition, the robot 1 has the telephone func- 
tion, the answering telephone, the call connecting func- 
tion, the remote control function, the security function, 
the automatically maintenance and version-up function, 
the E-mail management function, the helper-requiring- to 
person monitoring function, the dancing function, and 
the inter-robot communicating function, thereby making 
it possible to make the robot perform helpful movements 
and action for the user. Thus, a robot with high usability 
can be realized. 15 

(6) Other embodiments 

[021 0] Note that, in the aforementioned configuration, 
the case has been described, in which the appearance 20 
of the robot 1 is like a dog or cat so that a user can take 
to it similarly to real animals. However, the present in- 
vention is not limited thereto and other various kinds of 
appearances can be applied. In this case, plural rollers 
and wheels can be used in place of the plural legs 5A - 25 
5D as a moving means of the robot 1 . 
[0211] Further, in the aforementioned configuration, 
the case has been described, in which the robot 1 trans- 
mits and receives various kinds of information through 
the telephone line 44 to/from the outside. However, the 30 
present invention is not limited thereto and similar com- 
munication can be achieved through mobile telephone 
line, radio waves, a circuit for Internet, infrared signals, 
arLAN (Local Area Network), or the like. 
[0212] Furthermore, in the aforementioned configura- 35 
tion, various requests can be input by manipulating the 
key input section 24 or manipulating the dial buttons of 
the outside device 50. However, the present invention 
is not limited thereto and, in place of or in addition to it, 
a request can be input by voices using a sound recog- 40 
nizing function of the controller 20 or a request can be 
input with a remote controller. 

[0213] Furthermore, in the aforementioned configura- 
tion, as a method of manipulating an electric appliance 
in a house by the robot 1 , the case has been described, 
in which the robot 1 moves up to the electric appliances 
to output an infrared signal IR2 or to physically manip- 
ulate switches. However, the present invention is not 
limited thereto and a LAN can be provided in the house 
and all electric appliances in the house can be connect- so 
ed to this LAN so that the robot 1 can manipulate a cor- 
responding electric appliance through this LAN. Note 
that, in this case, the robot 1 robot 1 can move freely. 
[0214] Furthermore, in the aforementioned configura- 
tion, in the case of receiving a phone call or E-mail from 55 
the outside, the robot 1 moves up to the user. However, 
the present invention is not limited thereto and the robot 
1 can give different sounds depending on users or out- 



put the name of a corresponding user io inform who the 
phone call is for 

[0215] Furthermore, in the aforementioned configura- 
tion, the case has been described, where, in the case 
where connecting a phone call from the outside to a cor- 
responding user, the robot 1 looks for the user. However, 
the present invention is not limited thereto and, in the 
case where users are almost at fixed places, the robot 
1 can previously memorize the places of the users to 
move up to the place while confirming the posture and 
position using the output of the acceleration and angular 
velocity sensor 23 when receiving a phone call. 
[0216] Furthermore, in the aforementioned configura- 
tion, the case of memorizing sounds and images as an 
answering telephone function has been described. 
However, the present invention is not limited thereto 
and, for example, text data can be also memorized. In 
addition, the maintenance center 60 can manage text 
data such as E-mail, and, under the control of the main- 
tenance center 60, text such as E-mail can be written 
and read via the sound processor 21 of the robot 1 . 
[0217] Furthermore, in the aforementioned configura- 
tion, the case has been described, in which the robot 1 
looks for a user while confirming users when receiving 
a phone call from the outside. However, the present in- 
vention is not limited thereto and the robot 1 can give 
sounds from the speaker 11 to call the corresponding 
user when receiving a phone call and can moves its 
body toward the direction of hearing a response, or can 
move toward the corresponding user who performs 
movements (for example, waving his hand). 
[0218] Furthermore, in the aforementioned configura- 
tion, the case will be described, in which the robot 1 has 
a battery (a battery such as a primary battery, other than 
a repeatedly chargeable battery such as a lithium sec- 
ondary battery) as a power source. However, the 
present invention is not limited thereto and an alternat- 
ing power source for business can be used as a power 
source, for example. 

[0219] Furthermore, in the aforementioned configura- 
tion, in the explanation of the remote control function, 
the case where the robot 1 is controlled by manipulating 
the dial buttons of the outside device has been de- 
scribed. The present invention, however, is not limited 
thereto and a user gives a request, for example, "right" 
or "left", with sounds while confirming the surrounding 
situations which are photographed by the image recog- 
nizing section 12 and are transmitted from the robot 1 , 
and thereby the robot 1 can move in accordance with 
the request, has been described, in which the synchro- 
nized-signal generator 90 of the sound processor 21 is 
configured as shown in Fig. 26. However, the present 
invention is not limited thereto and for example, a judge- 
ment circuit is disposed at the input stage of the syn- 
chronized-signal generator 90 for judging whether an in- 
put signal is a MIDI signal and moreover, the synchro- 
nized-signal generator 90 can be configured so that the 
gathered signal S1 from the microphone 10, music in- 
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formation read out from a removable storage medium 
not shown, or music information such as MIDI signals 
are all supplied to this judgement circuit, MIDI signals 
are supplied to the MIDI signal processor 91 based on 
the judgement result of the judgement circuit, and non- 
MIDI signals are supplied to the BPF section 93. 
[0220] Furthermore, in the aforementioned configura- 
tion, the answering telephone function is applied to the 
robot 1 having the configuration as shown in Fig. 1. How- 
ever, the present invention is not limited thereto and it 
can be appliedto an information processing system hav- 
ing other various kinds of configurations. 
[0221] Furthermore, in the aforementioned configura- 
tion, the case has been described, in which a response 
message for answering telephone is produced by the 
sound processor 21 . However, the present invention is 
not limited thereto and such a response message can 
be previously stored in a prescribed storage medium like 
a memory and a corresponding message can be read 
out according to the user's conditions and be transmit- 
ted to a communicating party. 

Industrial Applicability 

[0222] A robot and an information processing system 
can be applied to an amusement robot, a helper robot 
and so on. 



Claims 

1. A robot comprising: 

a movement part for produce movements; 
an actuator for driving said movement part; 
an actuator control means for controlling said 
actuator; 

communicating means for receiving signals 
used between said robot and another electronic 
device; and 

information detecting means for detecting infor- 
mation from said signals received by said com- 
municating means, and wherein 
said actuator control means controls said actu- 
ator based on said information detected by said 
information detecting means, to make said 
movement part produce stated movements. 

2. The robot according to claim 1 , wherein: 

said communicating means is telephone com- 
municating means and further comprises send- 
er detecting means for detecting the sender of 
the signals received by said telephone commu- 
nicating means; and 

said actuator control means makes said move- 
ment part produce staled movement according 
to said detected sender. 



3. The robot according to claim 1 , wherein: 

said communicating means is telephone com- 
municating means andfurthercomprises send- 
5 er detecting means for detecting the sender of 

the signals received by said telephone commu- 
nicating means; and 

said actuator control means makes said move- 
ment part produce stated movement according 
10 to said detected sender. 

4. The robot according to claim 1, wherein: 
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said information detecting means is E-mail de- 
tecting means for detecting E-mail; and 
said actuator control means controls said actu- 
ator based on E-mail detected by said informa- 
tion detecting means to make said movement 
part produce stated movements. 

5. A robot comprising: 

memory means for memorizing stated informa- 
tion; 

memorized content changing means for chang- 
ing memorized content memorized by said 
memory means; 

communicating means for receiving signals 
used between said robot and another electronic 
device; and 

request detecting means for detecting an infor- 
mation updating request which indicates the 
update of said stated information, from the sig- 
nals received by said communicating means, 
and 

said memorized content changing means up- 
dates said stated information based on the in- 
formation updating request detected by said re- 
quest detecting means. 

6. The robot according to claim 5, wherein 

said stated information is control information 
for said robot. 



45 7. a robot comprising: 

output-of-sensor receiving means for receiving 
outputs from one or more sensor(s) provided in 
a user himself or near the user; 
so user's condition recognizing means for recog- 

nizing user's condition based on the outputs of 
said sensor(s) received by said output-of-sen- 
sor receiving means; 

a movement part for producing stated move- 
55 ments; 

an actuator for driving said movement part; 
an actuator control means for controlling said 
actuator; and 
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telephone communicating means for receiving 
signals used between said robot and another 
electronic device, and wherein 
when said telephone communicating means re- 
ceives a communication signal from another 
electronic device, if the phone call is not an- 
swered within a fixed time, said actuatorcontrol 
means makes said movement part produce 
stated movements based on the user's condi- 
tion recognized by said user's condition recog- 
nizing means. 

8. The robot according to claim 7, comprising: 

memory means for memorizing response mes- 
sages according to user's conditions; and 
response message output means for outputting 
said response messages from said memory 
means, and 

when saidtelephone communicating means re- 
ceives a communication signal from another 
electronic device, if the phone call is not an- 
swered within a fixed time, said response mes- 
sage output means outputs a response mes- 
sage corresponding to the user's condition rec- 
ognized by said user's condition recognizing 
means. 

9. An information processing system comprising: 

one or more sensor(s) provided in the user him- 
self or near the user; 

user's condition recognizing means for recog- 
nizing the user's condition based on the outputs 
of said sensor(s); 

response message memory means for memo- 
rizing response messages according to the us- 
er's conditions; and 

response message output means for outputting 
said response messages, and 
when a phone call is not answered within a fixed 
time, said response message output means 
outputs said response message based on the 
user's conditions recognized by said user's 
condition recognizing means. 

10. A robot comprising: 

a movement part for producing movements; 
an actuator for driving said movement part; 
actuator control means for controlling said ac- 
tuator; 

communicating means for transmitting signals 
used between said robot and another electronic 
device; 

information input means for inputting informa- 
tion from the outside; and 
information output means for outputting infor- 
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mation to the outside, and wherein: 

said actuator control means controls said 
actuator based on the information, which is 
input to said information input means, to 
make said movement part produce stated 
movements; and 

said information output means outputs 
stated information from said communicat- 
ing means to another electronic device 
based on the information, which is input to 
said information input means. 

11. A robot comprising: 

a sensor means for inputting information from 
a user; 

an movement part for producing movements; 
a actuator for driving said movement part: 
actuator control means for controlling said ac- 
tuator; 

information output means for outputting stated 
information; and 

communicating means for transmitting signals 
used between said robot and another electronic 
device, and wherein: 

when an input is not reached from a user 
within a fixed time, said actuator control 
means makes said movement part pro- 
duce stated movements; and 
after producing the stated movements, 
when an input is not reached yet from the 
user within a fixed time, said communicat- 
ing means transmits stated information, 
which is output from said information out- 
put means, to another electronic device. 

12. A robot comprising: 

a movement part for producing movements; 
an actuator for driving said movement part; 
actuator control means for controlling said ac- 
tuator; 

signal receiving means for receiving signals 
used between said robot and another electronic 
device; 

electronic device control means for controlling 
another electronic device; and 
information detecting means for detecting stat- 
ed information from the signals received by said 
signal receiving means, and wherein: 

said actuator control means controls said 
actuator based on the stated information 
detected by said information detecting 
means to make said movement part pro- 
duce stated movements; and 
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said electronic device control means con- 
trols another electronic device based on 
the stated information detected by said in- 
formation detecting means. 
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Brief Description of the Drawings 

1... robot, 2... body, 5A- 5D...leg, 6. ..head, 7... tail, 
10 - . .microphone , 11 ... speaker , 12... image recognizing section, 
13... image display, 20. . .controller, 21... sound processor, 
22. . .image processor, 24. . .key input section, 25- ..communication 
unit, 50. . .outside device, 60. . .maintenance center 
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